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Abstract 



'"" . . This self-contained paper is part of a series |FF2, |FF3| on actions by 

_^ diffcomorphisms of infinite groups on compact manifolds. The two main 

results presented here are: 

1. Any homomorphism of (almost any) mapping class group or auto- 
morphism group of a free group into Diff!j_(S' 1 ),r > 2 is trivial. For 
r — Nielsen showed that in many cases nontrivial (even faithful) 
representations exist. Somewhat weaker results are proven for finite 
index subgroups. 

00 . 2. We construct a finitely-presented group of real-analytic diffeomor- 

f^*) ■ phisms of R which is not residually finite. 

o 

1 Introduction 

O 

C^ ■ In this paper we consider infinite groups acting by diffeomorphisms on one- 

dimensional manifolds. For lattices in higher rank semisimple Lie groups such 
actions are essentially completely understood: 



in a 



S 

j^ ; Theorem 1.1 (Ghys [JGtjl , Burger-Monod jBMp . Let T be a lattice 

•*~j ■ simple Lie group ofR-rank at least two. Then any C° -action ofT on S 1 has a 

rS ' finite orbit, and any C 1 -action ofT on S 1 must factor through a finite group. 



Theorem 1.1 is the solution in dimension one of Zimmer's program of classify- 



ing actions of higher rank lattices in simple Lie groups on compact manifolds. In 



[La (see in particular §8), Labourie describes possible extensions of Zimmer's 



program to other "big" groups. A. Navas [Na has recently proven that any 
C 1+Q , a > 1/2 action of a group with Kazhdan's property T factors through a 
finite group. In this paper we consider three basic examples of nonlinear groups: 
mapping class groups of surfaces, (outer) automorphism groups of free groups, 
and Baumslag-Solitar groups. 

'Supported in part by the NSF and by the Sloan Foundation, 
t Supported in part by NSF grant DMS9803346. 



Mapping class groups and automorphism groups of free groups. Let 

Mod(g, k) denote the group of isotopy classes of diffeomorphisms of the genus g 



surface with k punctures. Unlike the case of lattices (Theorem 1.1), the group 
Mod(g, l),g > 1 does have a faithful, C°-action on S 1 without a global fixed 
point. This is a classical result of Nielsen. However, imposing a small amout of 
regularity changes the situation dramatically. 

Theorem 1.2 (Mapping class groups). For g > 3 and k = 0,1, any C 2 

action o/Mod(g, k) on S 1 or on I = [0, 1] is trivial. 



The S 1 case of Theorem 1.2 was announced by E. Ghys several years ago 



(see §8 of [ La | ) , and for real-analytic actions was proved by Farb-Shalen [FS|. 

Another class of basic examples of "big groups" are automorphism groups of 
free groups. Let Aut(F„) (resp. Out(F„)) denote the group of automorphisms 
(resp. outer automorphisms) of the free group F n of rank n. It is known that 



Aut(F„), n > 2 is not linear |FP 



The techniques we develop to prove the results above allow us to prove the 
following. 

Theorem 1.3 (Automorphism groups of free groups). For n > 6, any 

homomorphism from Aut(F n ) to Diff + (S' 1 ) factors through Z/2Z. Any homo- 
morphism from Aut(F n ) to Diff + (/) is trivial. The analogous results hold for 
Out(F„). 



M. Bridson and K. Vogtmann [BV| have recently proven a stronger result: 



any homomorphism from Aut(F n ) to a group which does not contain a sym- 
metric group must have an image which is at most Z/2Z. However, their proof 
uses torsion elements in an essential way, hence does not extend to finite index 
subgroups. On the other hand our techniques have implications for finite index 
subgroups of Aut(i ? „) and Mod(g, k); such subgroups are typically torsion free. 

Theorem 1.4 (Finite index and other subgroups). Let H be any group of 
C 2 diffeomorphisms of I or S 1 with the property that no nontrivial element of 
H has an interval of fixed points (e.g. H is a group of real- analytic diffeomor- 
phisms). Then H does not contain any finite index subgroup of: 

1. MoA{g,k) for g>3,k> 0. 

2. Aut(F„) or Out(F„) for n > 6. 

3. The Torelli group T 3i k for g > 3, k > 0. 



In FFS] we construct a C 1 action of T g ,k on / and on S 1 . Note also that 
Out(i<2) has a free subgroup of finite index, which admits a faithful, C" action 
on / and on S 1 . 

Baumslag-Solitar groups. The Baumslag-Solitar groups BS(m, n) are de- 
fined by the presentation 

BS(m, n) =<a,b: ab m a - 1 = b n > 



When n > m > 1 the group BS(m,n) is not residually finite; in particular 
it is not a subgroup of any linear group (see, e.g. ]L"S|). We will give a con- 
struction which shows that BS(to, n) is a subgroup of one of the "smallest" 
infinite-dimensional Lie groups. 

Theorem 1.5 (Baumslag-Solitar groups: existence). The group Diff " (R) 
of real- analytic diffeomorphisms o/R contains a subgroup isomorphic to BS(m, n) 
for any n > m > 1. TTie analogous result holds /or Homeo+(5 1 ) and Homeo + (/) . 

It is not difficult to construct pairs of diffeomorphisms a, b £ Diff^(R) which 
satisfy the relation ab m a~ 1 = b n ; the difficulty is to prove that (in certain 
situations) this is the only relation. To do this we use a Schottky type argument. 

While Ghys-Sergiescu ]Gq] showed that Diff°°(5 1 ) contains Thompson's in- 
finite simple (hence non-residually finite) group T, they also showed that T 



admits no real-analytic action on S 1 (see also [FS|); indeed we do not know of 



any subgroups of Diff"(5' 1 ) which are not residually finite. 



The construction in the proof of Theorem 1.5 gives an abundance analytic 
actions of BS(m, n) on R, and C° actions of BS(m, n) on S 1 and on /. The loss 
of regularity in moving from R to S 1 is no accident; we will show in contrast to 
Theorem |1.5| that (for typical m, n) there are no C 2 actions of BS(m, n) on S 1 
or /. 

Theorem 1.6 (Baumslag-Solitar groups: non-existence). No subgroup of 
Diff + (/) is isomorphic to BS(m, n),n > m > 1. If further m does not divide n, 
then the same holds for Diff + (5' 1 ). 

The hypothesis m > 1 in Theorem |1.6| is also necessary since BS(1, n),n > 1 
is a subgroup of PSL(2, R), hence of Diff^S 1 ). 

In fact BS(l,n) has many actions on S , and it is natural to attempt a 
classification (up to conjugacy) of all of them. As a first step in this direction 



we observe (Theorem jxJI), as a corollary of result of M. Shub [ 3h| on expanding 
maps, that the standard, projective action of BS(l,n) on S 1 is locally rigid, i.e. 
nearby actions are conjugate. An example due to M. Hirsch |HJ shows that it is 
not globally rigid, i.e. there are actions which are not conjugate to the standard 
action (Theorem pjl below). It would be interesting to find a numerical invariant 



which characterizes the standard action, as Ghys |Gh2| has done for Fuchsian 
groups. 

2 Tools 

In this section we recall some properties of diffeomorphisms of one-manifolds 
which will be used throughout the paper. 

2.1 Kopell's Lemma and Holder's Theorem 

Our primary tool is the following remarkable result of Nancy Kopell which is 
Lemma 1 of [BO. 



Theorem 2.1 (Kopell's Lemma). Suppose f and g areC 2 , orientation-preserving 
diffeomorphisms of an interval [a, b) such that fg — gf. If f has no fixed point 
in (a, b) and g has a fixed point in (a, b) then g = id. 

Another useful result is the following theorem, which is classical. 

Theorem 2.2 (Holder's Theorem). Suppose a group G of homeomorphisms 
of R acts freely and effectively on a closed subset of R. Then G is abelian. 

2.2 The translation number and mean translation number 

If / £ Homeo+(5 1 ), i.e, / is an orientation preserving homeomorphism of S , 
then there is a countable collection of lifts of / to orientation preserving home- 
omorphisms of the line. If F is one such lift then it satisfies FT — TF where 
T(x) = x + 1, and all others are of the form FT n , n £ Z. Any orientation 
preserving homeomorphism of R which commutes with T is a lift of an element 
of Homeo+(S' 1 ). We will denote by Homeoz(R) the group of homeomorphisms 
of R which commute with T, or equivalently the group of all lifts of elements 
ofHomeo + (S' 1 ). 

There are two important and closely related functions from Homeo+(<S' 1 ) 
and Homeoz(R) to S 1 and R respectively, which we now define. 

Definition 2.3. If F £ Homeoz(R) define its translation number t(F) £ R 
by 

r(F) = lim *"(*>-* 

If f G Homeo+(5' 1 ) define its rotation number, p(f) £ S , by 

p(f)=r(F) mod(l), 
where F is any lift of f . 

We summarize some basic properties of the rotation and translation num- 
bers. Proofs of these and additional properties can be found, for example, in 



I ftMvS l 



Proposition 2.4 (Properties of rotation number). IfF £ Homeoz(R) the 
number t(F) always exists and is independent of the x £ R used to define it. It 
satisfies 

r(F") = n-T(F) 
T{FT n ) = t(F) + n 
r(F Fi^ 1 ) = t(F) for any F a £ Homeoz(R). 

F has a fixed point if and only if t(F) = 0. 



If f £ Homeo+(S' 1 ) then p(f) is independent of the lift F used to define it. 
It satisfies 

P(f n ) = np(f) 
Pifoffo 1 ) = P(f) f° r an V /o e Homco + (S' 1 ). 

/ has a fixed point if and only if p{f) = 0. 

Suppose G is a subgroup of Homeo+(S' 1 ) and G is the group of all lifts of 
elements of G to R. If G preserves a Borel probability measure po on S 1 then 
this measure may be lifted to a G invariant measure /jonR which is finite on 
compact sets and which is preserved by the covering translation T(x) = x + 1. 
This permits us to define the mean translation number with respect to fi. 

Definition 2.5. If F G Homeoz(R) define its mean translation number t^(F) G 
R by 

U([x,F(x))) ifF(x)>x, 

T li {F)=iO ifF(x)=x, 

[-n([F(x),x)) ifF(x)<x, 

where x G R. 

We enumerate some of the well known basic properties of the mean transla- 
tion number which we will use later. 

Proposition 2.6. Suppose G is a subgroup of Homeo+(S' 1 ) which preserves 
the Borel probability measure and G is the group of all lifts of elements of G. If 
F G G the mean translation number t^(F) is independent of the point x used 
to define it. Indeed t^(F) = t(F). Moreover the function r M : G — > R is a 
homomorphism (and hence so is t : G — > R.) 

Proof. Consider the function 

(mO,?/))) ify>x, 
if y = x, 

-p{[y, x )) tty<x. 

It has the property that v{x, y) + v(y, z) = v(x, z). 

We also note that t^(F) = v(x 1 F(x). To see this is independent of x note 
for any y G R, v[x, F{x)) = v(x, y) + u(y, F(y)) + v(F(y), F{x)) = u(y, F{y)) + 
v(x,y) — is(F(x),F(y)). But F preserves the measure fi and the orientation of 
R so u(x, y) = u(F(x),F(y)). Hence v(x, F(x) = u(y, F(y)). 

Since p: is the lift of a probability measure on S 1 we know that p,(\x, x + k)) = 
k for any k G Z. So if F n (x) G [x + k, x + k + 1) we see that 

F n (x) -x-l<k< v(x, F n (x)) < k + 1 < F n (x) -x + 1. 



To see that t^(F) = t(F) we note 

r(F) = ton F " 1{X) ~ X 

n— >oo fi 

n — >oo fl 

n-1 



,0 ^ 

lim -Vi/(P(4F i+1 (i)) 

i. — >rxn r) ' ^ 



n — >oc Jl 

i=0 

v{x,F{x)) 



There are two well known corollaries of this result which we record here for 
later use. 

Corollary 2.7. Suppose G is a subgroup of Homeo+(S' 1 ) which preserves the 
Borel probability measure pa and G is the group of all lifts of elements of G. 
Then each of the functions t m : G — > R, r : G — * R and p : G — * S 1 is a 
homomorphism. 

Proof. To see that r^ : G — ► R is a homomorphism we suppose f,g G G 
and consider v as defined above. Then T^(fg) — v{x, f(g(x))) = v(x,g(x)) + 
is(g(x)J(g(x)))==T^g)+T^f). 

Since Tn = r we know that r is also a homomorphism. There is a natural 
homomorphism 7r : G — > G which assigns to g its projection on 5 1 . If p : R — » 
S 1 = R/Z is the natural projection then for any f,g G G, 

pW/s)) =p(r(fg)) =p(r(/)) +^(.9)) = p(tt(/)) +p(7r( ff )). 
Hence p : G — > 5 1 is a homomorphism. o 

Corollary 2.8. If G is an abelian subgroup o/Homeo + (S' 1 ) and G is the group 
of all lifts to R of elements of G, then G is abelian and both r : G — > R and 
p : G ^ S 1 are homomorphisms. 

Proof. Since G is abelian it is amenable and there is a Borel probability measure 
po on S 1 invariant under G Let p be the lift of this measure to R. Then 



Corollary 2.7 implies that r : G — > R and p : G ^ S 1 are homomorphisms. 

As above let 7r : G — > G be the natural projection. The kernel of it is 
{T n }„ eZ , where T(x) — x + 1, i.e., the lifts of the identity. If f,g G G then 
[/, 5] is in the kernel of tt so [/, g] = T k , for some k G Z. But since p is a 
homomorphism /?([/, 5]) = which implies k — 0. Hence G is abelian. o 

The following easy lemma turns out to be very useful. 



Lemma 2.9. Suppose G is a subgroup of Homeo+(5' 1 ) which preserves the 
Borel probability measure /^o and /o,3o S G satisfy p(fo) = p(.9o)- Then fo(x) = 
go(x) for all x in the support of /j,q. 

Proof. Let /x be the lift of the measure [io to R. Pick lifts / and g of /o and 
go respectively which satisfy Tp(f) = Tp(g). Then for any x s R we have 

M([*>/(a:))) = t m(/) = v(#) =A*([a ; »fl , (a;)))- 

Suppose that g(x) > /(x). It then follows that /«([/(#), <7(x)) = 0, so for 
a sufficiently small e > 0, n([f(x),f(x) + e]) = and /i([g(x) — e,g(x)j) = 0. 
Applying gf^ 1 to [f(x),f(x) + e) we see that fi(\g{x),g(x) + e')) = 0. Hence 
fj,([g(x) — e,g(x) + e')) = which is not possible if x and hence g{x) is in the 
support of /i. 

We have shown that x £ supp(^) implies f(x) < g(x). The inequality g{x) < 
f{x) is similar. Projecting back to S 1 we conclude that /q = go on su PP(/ i o)- ° 

One additional well known property we will use is the following. 

Proposition 2.10. If g G Homeo + (5 1 ) is an irrational rotation then its cen- 
tralizer Z(g) in Homeo+(S' 1 ) is the group of rigid rotations of S 1 . 

Proof. Let / be an element of Z(g). Then the group generated by / and g 
is abelian and hence amenable so there it has an invariant Borel probability 
measure. But Lebesgue measure is the unique Borel probability measure in- 
variant by the irrational rotation g. Since / preserves orientation and Lebesgue 
measure, it is a rotation, o 



3 Mapping Class Groups 

In this section we prove Theorem |l.2|. Several of the results, in particular those 



in §3.1, will also be used to prove Theorem 1.3 



3.1 Fully Supported Diffeomorphisms 

One of the main techniques of this paper is to use Kopell's Lemma (Theorem 



2.1) to understand actions of commuting diffeomorphisms. The key dichotomy 
that arises is the behavior of diffeomorphisms with an interval of fixed points 
versus those which we call fully supported. 

If / is a homeomorphism of a manifold M, we will denote by <9Fix(/) the 
frontier of Fix(/), i.e. the set <9Fix(/) = Fix(/) \ Int(Fix(/)). 

Definition 3.1 (Fully supported homeomorphism). A homeomorphism f 
of a manifold M is fully supported provided that Int(Fix(/)) = 0, or equivalently 
<9Fix(/) = Fix(/). A subgroup G of Homeo(M) will be called fully supported 
provided that every nontrivial element is fully supported. 



It is a trivial but useful observation that if M is connected then for every 

nontrivial homeomorphism / of M, the set Fix(/) ^ if and only if <9Fix(/) ^ 0. 

The following lemma indicates a strong consequence of Kopell's Lemma 



(Theorem 2.1) 



Lemma 3.2 (Commuting diffeomorphisms on I). Suppose f and g are com- 
muting orientation preserving C 2 diffeomorphisms of I. Then f preserves every 
component ofFix(g) and vice versa. Moreover, dFix(f)) C Fix(g) and and vice 
versa. In particular if f and g are fully supported then Fix(/) = Fix(g). 

Proof. The proof is by contradiction. Assume X is a component of Fix(g) and 
f(X) 7^ X. Since / and g commute f(Fix(g)) = Fix(gr) so f(X) / X implies 
f(X) n X = 0. Let x be an element of X and without loss of generality assume 
f(x) < x. Define 

a = lim f n (x) and b = lira f n (x). 

n — >oo n — > — oo 

Then a and b are fixed under both / and g and / has no fixed points in (a, b). 



Then Kopell's Lemma (Theorem 2.1) implies g(y) = y for all y £ [a, 6], contra- 
dicting the hypothesis that X is a component of Fix(g). The observation that 
<9Fix(/) C Fix(g) follows from the fact that x £ 9Fix(/) implies that either 
{x} is a component of Fix(/) or x is the endpoint of an interval which is a 
component of Fix(/) so, in cither case, x € Fix(g). o 



There is also a version of Lemma 3.2 for the circle. For g £ Homeo+(S' 1 ) let 



Per(g) denote the set of periodic points of g. 

Lemma 3.3 (Commuting diffeomorphisms on S 1 ). Suppose f and g are 
commuting orientation preserving C 2 diffeomorphisms of S 1 . Then f preserves 
every component of Per (g) and vice versa. Moreover, <9Per(/) C Pcr(g) and 
vice versa. In particular if neither Per(/) or Pcr(g) has interior then Per(/) = 
Per(.g). 

Proof. First consider the case that Per(g) = 0. The assertion that / preserves 
components of Per(g) is then trivial. Also, g is topologically conjugate to an 



irrational rotation which by Proposition 2.10 implies that / is topologically 
conjugate to a rotation and hence Per(/) = or Per(/) = S 1 . In either case we 
have the desired result. 

Thus we may assume both / and g have periodic points. Since for any circle 
homeomorphism all periodic points must have the same period, we can let p be 
the least common multiple of the periods and observe that Per(/) = Fix(/ P ) 
and Pcr(g) = Fix(g p ). Since / and g commute and both f p and g p have fixed 
points, if x G Fix(/ P ) then y = linin^oo g np (x) will exist and be a common 
fixed point for f p and g p . If we split the circle at y we obtain two commuting 
diffeomorphisms, f p and g p , of an interval to which we may apply Lemma p.2| 
and obtain the desired result, o 



Lemma 3.4. Let go G DifF + (S' 1 ) and let Z{go) denote its centralizer. Then the 
rotation number p : Z(go) — > S 1 is a homomorphism. 

Proof. We first observe the result is easy if go has no periodic points. In this case 
go is conjugate in Homeo+(S' 1 ) to an irrational rotation by Denjoy's Theorem, 
and the centralizer of an irrational rotation is abelian by Proposition 2. 10|. So 



the centralizer of g in Diff + (S fl ) is abelian. But then p restric ted to an abelian 



subgroup of Homeo+(S' 1 ) is a homomorphism by Proposition 2.g . 

Thus we may assume go has a periodic point, say of period p. Then ho = ffg 
has a fixed point. Let He a lift of ho to R which has a fixed point. Let G 
denote the group of all lifts to R of all elements of Z(g ). Note that every 



element of this group commutes with h because by Corollary 2.5 any lifts to R 
of commuting homeomorphisms of S 1 commute. 

Let X = dFix(h) and observe that g(X) — X for all g G G, so G acts on 
the unbounded closed set X and G acts on Xo = dFix(ho) which is the image 
of X under the covering projection to S 1 . We will show that if Fix(g) flX^I 
then X C Fix(g). Indeed applying Lemma |3.3| to ho and map <?i of S 1 which g 
covers, we observe that dFix(h) C Fix(gi) which implies X C <9Fix(<?) if g has 
has a fixed point. 

It follows that if H = {g € G \g(x) = x for all x G X} is the stabilizer of 
X then G/H acts freely on X and hence is abelian by Holder's theorem. Also 
G/H acts on X and hence there is a measure po supported on X and invariant 
under G/H. Clearly this measure is also invariant under the action of Z(g ) on 
S . The measure po lifts to a G-invariant measure p. 



It follows from Corollary 2.7 that the translation number r : G — > R and 



the rotation number p : Z{go) — * S 1 are homomorphisms. o 

The commutativity graph of a set S of generators of a group is the graph 
consisting of one vertex for each s G S and an edge connecting elements of S 
which commute. 

Theorem 3.5 (abelian criterion for /). Let {gi,g2, ■ ■ ■ ,gk} be a set of fully 
supported elements o/Diffi (/) and let G be the group they generate. Suppose 
that the commutativity graph of this generating set is connected. Then G is 
abelian. 



Proof. By Lemma 3.2 we may conclude that for each ji,J2 we have Fix(^ 1 ) = 
Fix(<? j2 ). Call this set of fixed points F. Clearly F is the set of global fixed 
points of G. 

Fix a value of i and consider Z{gi). Restricting to any component U of the 
complement of F we consider the possiblity that there is an h G Z(gi) with a 



fixed point in U. Kopell's Lemma (Theorem 2.1), applied to the closure of the 
open inteval U, tells us that such an h is the identity on U. Thus the restriction 
of Z(gi) to U is free and hence abelian by Holder's Theorem. But U was an 
arbitrary component of the complement of F and obviously elements of Z(gi) 
commute on their common fixed set F. So we conclude that Z(gi) is abelian. 



We have hence shown that if gj and gt are joined by a path of length two in 
the commutativity graph, they are joined by a path of length one. A straight- 
forward induction shows that any two generators are joined by a path of length 
one, i.e. any two commute, o 



There is also a version of Theorem 3.5 for the circle 



Corollary 3.6 (Abelian criterion for S 1 ). Let {<?i, <72> • • • > <?&} be a set of 

fully supported elements of Diff + (5' 1 ), each of which has a fixed point, and let 
G be the group they generate. Suppose that the commutativity graph of these 
generators is connected. Then G is abelian. 



Proof. By Lemma 3.3 we may conclude that for each j, k we have Pei(gj) = 
Per(gfc). But since each gi has a fixed point Per(gi) — Fix(gi). Hence there is a 
common fixed point for all of the generators. Splitting S 1 at a common fixed 
point we see G is isomorphic to a subgroup of Diff , (I) satisfying the hypothesis 



of Theorem 3.5. It follows that G is abelian. o 



Question. Let G be a subgroup of Homeo+(5' 1 ). If the commutativity graph 
for a set of generators of G is connected, does this imply that rotation number 
is a homomorphism when restricted to G? 

We will need to understand relations between fixed sets not just of commut- 
ing diffeomorphisms, but also of diffeomorphisms with another basic relation 
which occurs in mapping class groups as well as in automorphism groups of free 
groups. 

Lemma 3.7 (aba lemma). Suppose a and b are elements o/Homeo+(/) or 
elements of Homeo+(S' 1 ) which have fixed points. Suppose also that a and b 
satisfy the relation a ni b m3 a n2 = b mi a n3 b m2 with m 1 + m 2 / m 3 . If J is a 
nontrivial interval in Fix(a) then either J C Fix(fe) or J D Fix(fe) = 0. 

Proof. Suppose J C Fix(a) and suppose z G J fl Fix(&). We will show that this 
implies that J C Fix(6). 

Let x £ J. Observe that a ni b m3 a n2 = b mi a n3 b m2 implies a~ n2 &- m3 a -ni = 
b~ m2 a~ n3 b~ mi . Hence we may assume without loss of generality that b m3 (xo) 
is in the subinterval of J with endpoints Xq and z (otherwise replace a and b by 
their inverses). In particular b m3 (xo) £ J. Note that 

b m3 {x ) = a ni b m3 {x ) = a ni b m3 a n2 {x ) = b mi a n3 b m2 (x ) = b mi b m2 { X() ) = b mi+rn2 (x ) 

This implies 6 mi+m2_m3 (a;o) = ^o an( i since mi +m 2 — m 3 ^ we can conclude 
that b(xo) = xq- Since xq G J was arbitrary we see J C Fix(&). o 
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3.2 Groups of type MC(n) 

We will need to abstract some of the properties of the standard generating 
set for the mapping class group of a surface in order to apply them in other 
circumstances. 

Definition 3.8 (Groups of type MC(n)). We will say that a group G is of 
type MC(n) provided it is nonabelian and has a set of generators {di,&i}™ =1 U 
{cj}^Zi with the properties that 

1. aittj — ajai, bibj — bjbi, c%Cj — CjCi, aiCj — Cjdi, for all i,j, 

2. a 2 bj = bjdi, if i ^ j, 

3. focj = Cjh, if j ^i,i — l, 

4- aibiai = biaibi for 1 < i < n, and biCjbi = CjbiCj whenever j — i,i — 1. 

A useful consequence of the MC(n) condition is the following. 

Lemma 3.9. If G is a group of type MC(n) and {a i ,b i } 7 j L 1 U {cj}™~1 are the 
generators guaranteed by the definition then any two of these generators are 
conjugate in G. 

Proof. The relation aba = bab implies a = {ab)~ l b(ab), so a and b are conjugate. 
Therefore property (4) of the definition implies a^ is conjugate to bi and 6j is 
conjugate to Cj and Cj_i. This proves the result, o 

The paradigm of a group of type MC(n) is the mapping class group Mod(n, 0) . 

Proposition 3.10 (Mod(n, 0) and Mod(n, 1)). For n > 2 and k = 0, 1 the 

group Mod(n, k) contains a set of elements 

S={a l MUU{c J } n J Zl 
with the following properties: 

1. For k = the set S generates all of Mod(n, 0). 

2. The group generated by S is of type MC(n). 

3. There is an element of g G Mod(n, k) such that g~ l a\g — a n , g _1 b\g = 
b n , g~ 1 b 2 g = b n -i, and g^c^g = c„_i. 

4- If Go is the subgroup ofMod(n,k) generated by the subset 

then Go is of type MC(n — 1). 

Proof. Let S denote the surface of genus n with k punctures. Let i(a, [3) denote 
the geometric intersection number of the (isotopy classes of) closed curves a 
and [3 on S. Let ai,bi,Cj with 1 < i < n, 1 < j < n be Dehn twists about 
essential, simple closed curves a{,f3i,jj with the following properties: 
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• 



• 



i(an,aj) = i{Pi,Pj) = i{n,lj) = for i ^ j. 
i(ai, ft) = 1 for each 1 < i < n. 

• *(7j> &) = 1 = i(7j! /5j+i) IOT 1 < i < » - 1< 

It is known (see, e.g.W) that there exist such <n,bi,Cj which generate 
Mod(n,0); we choose such elements. These are also nontrivial in Mod(n, 1). 
Since Dehn twists about simple closed curves with intersection number zero 
commute, and since Dehn twists a, b about essential, simple closed curves with 
intersection number one satisy aba = bab (see, e.g. Lemma 4.1.F of (lv[), prop- 



erties (l)-(4) in Definition 3.8 follow. We note that the proofs of these relations 
do not depend on the location of the puncture, as long as it is chosen off the 
curves a*, /%, jj . 

To prove item (2) of the lemma, consider the surface of genus g — 2 and 
with 2 boundary components obtained by cutting E along ot\ and 71. The 
loop Pi becomes a pair of arcs connecting 2 pairs of boundary components, and 
P2 becomes an arc connecting 2 boundary components. The genus, boundary 
components, and combinatorics of arcs is the exact same when cutting E along 
a n and 7„_i. Hence by the classification of surfaces it follows that there is 
a homeomorphism between resulting surfaces, inducing a homeomorphism h : 
E — > E with h(ai) = a n ,h(~/i) = 7„_i and h(0i) = f3 n -i+x,i = 1,2. The 
homotopy class of h gives the required element g. 

To prove item (3), let r be an essential, separating, simple closed curve on 
E such that one of the components E' of E — r has genus one and contains a n 
and b n . Then there is a homeomorphism h G Homeo + (E) taking any element of 
{<Zj, bi}™~^ U{cj}™~f to any other element, and which is the identity on E'. Then 
the isotopy class of h, as an element of Mod(n, 0) , is the required conjugate, since 
it lies in the subgroup of Mod(0, n) of diffeomorphisms supported on E — E', 
which is isomorphic to Mod(n — 1, 1) and equals Gq. o 



3.3 Actions on the interval 

In this subsetion we consider actions on the interval / = [0, 1]. 

Theorem 3.11 (AfC(n) actions on /). Any C 2 action of a group G of type 
MC(ri) for n > 2 on an interval I is abelian. 

Proof. We may choose a set of generators {at, bi}f =l U {cj}""!^ for G with the 



properties listed in Definition 3.8 



If G has global fixed points other than the endpoints of / we wish to consider 
the restriction of the action to the closure of a component of the complement of 
these global fixed points. If all of these restricted actions are abelian then the 
original action was abelian. Hence it suffices to prove that these restrictions are 
abelian. Thus we may consider an action of G on a closed interval Iq with no 
interior global fixed points. None of the generators above can act trivially on 
Iq since the fact that they are all conjugate would mean they all act trivially. 
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We first consider the case that one generator has a nontrivial interval of fixed 
points in Jo and show this leads to a contradiction. Since they are all conjugate, 
each of the generators has a nontrivial interval of fixed points. 

Choose an interval J of fixed points which is maximal among all intervals of 
fixed points for all of the dj. That is, J is a nontrivial interval of fixed points 
for one of the aj, which we assume (without loss of generality) is a 2 , and there 
is no J 1 which properly contains J and which is pointwise fixed by some dj. 

Suppose J = [xq, x{\. At least one of xo and X\ is not an endpoint of Jo since 
a,2 is not the iden tity. Suppose it is Xq which is in Int(7o). Then Xo G <9Fix(a2), 



so by Lemma 3.2 we know that xq is fixed by all of the generators except possibly 
62, since 62 is the only generator with which 02 does not commute. The point 
xo cannot be fixed by 62 since otherwise it would be an interior point of Iq fixed 
by all the generators, but there are no global fixed points in Iq other than the 



endpoints. The identity 02^202 = 62^2^2 together with Lemma 3.7 then tells us 
that b 2 (J) n J = 0. 

Now assume without loss of generality that 62(^0) > ^o (otherwise replace 
all generators by their inverses). Define 

yo = lim 62(0:0) an(i V 1 = lim ^2(^0)- 

n — > — 00 n — >oc 

Since bi(J) H J = we have b^iJ) (~l J = and hence yo < x < x\ < y\. But 
Xo is a fixed point of a\ (as well as the other generators which commute with 02) 
and ai commutes with bi. Hence yo and y\ are fixed by a\ ( since 62 preserves 



Fix(ai)). We can now apply Kopell's Lemma (Theorem 2.1 ) to conclude that 
<Xi is the identity on [yo,y\] which contradicts the fact that J — [xo,Xi] is a 
maximal interval of fixed points among all the a^. 

We have thus contradicted the supposition that one of the generators has 
a nontrivial interval of fixed points in Iq, so we may assume that each of the 
generators, when restricted to Io, has fixed point set with empty interior. That 
is, each generator is fully supported on Iq. 

We now note that given any two of the generators above, there is another 
generator h with which they commute. Thus we may conclude from Lemma |3.5| 
that the action of G on Iq abelian. o 



Proof of Theorem 1.2 for I: By Proposition 3.10, the mapping class group 
Mod(g, fc), g > 2, k — 0, 1 is a group of type MC{g). Since every abelian quotient 
of Mod(g, k),g > 2 is finite (see, e.g. Q), in fact trivial for g > 3, and since 



finite groups must act trivially on I, the statement of Theorem 1.2 for I follows. 



3.4 Actions on the circle 



We are now prepared to prove Theorem 1.2 in the case of the circle S 1 



Theorem [l.2j (Circle case). Any C 2 action of the mapping class group 
M(g, k) of a surface of genus g > 3 with k = 0, 1 punctures on S 1 must be 
trivial. 
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As mentioned in the introduction, the C 2 hypothesis is necessary. 

Proof. Closed case. We first consider the group G — Mod(<7, 0), and choose 
generators {ai,&i}™ =1 U {cj}" 1 !^ for G with the properties listed in Definition 



3.8 and Proposition 3.10|. All of these elements are conjugate by Lemma 3.9 



so 



they all have the same rotation number. 

We wish to consider a subgroup Go of type MC(n — 1). We let u = a^ 1 and 
for 1 < i < n — 1 we set Ai = aiU and Bi = biU. For 1 < i < n — 2 we let 
Ci = ciu. Since u commutes with any element of {ai, bi}™^ U {cj}™Zi we have 
the relations 

1. A^Aj = AjAi, Ditij = iSjijj, GiLjj = GjL>i, AiGj = GjA$, tor all i,J, 

2. AiBj = Bj A h Hi ^ j, 

3. B i C j = C j B i ,iij^i,i + l, 

4. AiBiAi = BiAiBi and BiCjBi = CjBiCj whenever j = i, i — 1. 

We now define Go to be the subgroup of G generated by {At, Bi}"~^ U 
{Gy}™"- 2 . The fact that u commutes with each of {a i: bi}^~^ U {cj}™" 2 and has 
the opposite rotation number implies that the rotation number of every element 
of {Ai, Bi}™~^ U {Gj}™r 2 is 0. Thus each of these elements has a fixed point. 

We note that given any two of these generators of Go there is another gen- 
erator with which they commute. If all of these generators have fixed point sets 



with empty interior then we may conclude from Lemma 3.2 that any two of 
them have equal fixed point sets, i.e. that Fix(A;) = Fix(Bj) = Fix(Gfe) for all 
i,j,k. So in this case we have found a common fixed point for all generators of 
Go- If we split S 1 at this point we get an action of Go on an interval / which 



must be abelian by Theorem 3.11. It follows that all of the original generators 
{a,i,bi}™ =1 U {cj}^Z{ commute with each other except possibly c„_i may not 
commute with b n ^\ and b n , and a n and b n may not commute. 

Let if) : G — > Diff 2 ^(»S' 1 ) be the putative action, and consider the element g 



guaranteed by Proposition 3.10J . We have that 



and 

[</>(€„_!), <MM] = Mgy'McJMhMig) =%d 

and 

[V>(c„-i),V(&n-i)] = iig)- 1 mc 1 ),^(b 2 )]iP(g) = id 

Hence G is abelian, hence trivial since (see Q) abelian quotients of Mod(<7, k),g > 
3 are trivial. 

Thus we are left with the case that one generator of Go has a nontrivial 
interval of fixed points in S 1 . Since they are all conjugate, each of the generators 
of Go has has a nontrivial interval of fixed points. Also we may assume no 
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generator fixes every point of S 1 since if one did the fact that they are all 
conjugate would imply they all act trivially. 

Choose a maximal interval of fixed points J for any of the subset of generators 
{Ai}. That is, J is a nontrivial interval of fixed points for one of the {Ai}, which 
we assume (without loss of generality) is A2, and there is no J' which properly 
contains J and which is pointwise fixed by some Aj. 

Sup pose xq and X\ are the endpoints of J. Then x$,x\ S 9Fix(^2) so by 



Lemma 3.3, xq and x\ are fixed points for all of the generators except B2, since 
i?2 is the only one with which A2 does not commute. If the point xo 6 Fix(i?2) 
we have found a common fixed point for all the generators of Go and we can 
split S 1 at this point obtaining an action of Go on an interval which implies Go 



is abelian by Theorem 3.11 . So suppose xq ^ Fix(i?2). 

Recall that the diffeomorphism B2 has rotation number 0, so it fixes some 
point. Thus the identity A2B2A2 = B2A2B2, together with the fact that B2 



cannot fix Xo, implies by Lemma Qj\ that B 2 (J) fl J — and J n B 2 1 (J) = 0. 
Define 

yo = lim B2 (xo) and y\ = lim B%(xo). 

n — > — 00 n — >oo 

We will denote by K the interval in S 1 with endpoints j/o & n d yi which contains 
xq. Note that K properly contains J and that B2(K) = K. 

Since xq G Fix(A\) and A\ commutes with B2 we conclude yo and y\ are 



fixed by A\. We can now apply Kopcll's Lemma (Theorem 2.1) to the interval 
K with diffcomorphisms B2 and A\ to conclude that A\ is the identity on K. 
But J is a proper subinterval of K which contradicts the fact that J was a 
maximal interval of fixed points for any one of the A4. Thus in this case too we 
have arrived at a contradiction. 

Punctured case. We now consider the group Mod(g, 1), and choose elements 
{ai, biYi=i u { c j} 1 jZi as above. The argument above shows that the subgroup G 
of Mod(g, 1) generated by these elements [] acts trivially on S 1 . We claim that 
the normal closure of G in Mod(,<?, 1) is all of Mod(g, 1), from which it follows 
that Mod(g, 1) acts trivially, finishing the proof. 

To prove the claim, let S g denote the closed surface of genus g and recall the 
exact sequence (see, e.g. fry)]): 

1 -> iCi(S g ) -> Mod(g, 1) -> Mod( 3 , 0) -y 1 

where iri(Sigma g ) is generated by finitely many "pushing the point" homco- 
morphismspr around generating loops r in m(S g ) with basepoint the puncture. 
Each generating loop r has intersection number one with exactly one of the 
loops, say /3j, corresponding to one of the twist generators of G. Conjugating bi 
by p T gives a twist about a loop f3[ which together with (3i bounds an annulus 
containing the puncture. The twist about (3i composed with a negative twist 

x We do not know wether or not G actually equals Mod(g, 1); there seems to be some 
confusion about this point in the literature. 
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about 13[ gives the isotopy class of p T . In this way we see that the normal closure 
of G in Mod(g, 1) contains G together with each of the generators of the kernel 
of the above exact sequence, proving the claim, 
o 



4 Aut(F n ), Out(F n ) and other subgroups 



In §]4.1| we prove Theorem |1.3[ Note that the result of Bridson-Vogtmann men- 
tioned in the introduction implies this theorem since it is easy to see that any 
finite subgroup of Homco+(S' 1 ) is abclian so their result implies ours. We give 
our proof here because it is short, straightforward and provides a good illustra- 
tion of the use of the techniques developed above. While several aspects of the 
proof are similar to the case of Mod(g, 0), we have not been able to find a single 
theorem from which both results follow. 



In §4.2 we prove theorem 1.4, extending the application to finite index sub- 



groups of Mod(g, k) and Aut(F n ). 

4.1 Actions of Aut(F n ) and Out(F n ) 

We begin with a statement of a few standard facts about generators and relations 
of Aut(F„). 

Lemma 4.1 (Generators for Aut(F n )). The group Aut(F„) has a subgroup 
of index two which has a set of generators {A^, B^} with i =/= j, 1 < i < n and 
1 fs j ' < n - These generators satisfy the relations 

A tj A ki = AkiAtj and B tJ B M = B kl B V] if {i, j} n {k, = (i) 

[Aij , A jk ] = A~ k x and [Aij , Aj k l ] = A lk (ii) 

[Bij , Bjk] = B^ 1 and [B tJ , B^ 1 } = B lk (iii) 

AjAj^Aij = A^AijAjl and B^By = Br^B]? (iv) 

Proof. Let {ei}^ =1 be the generators of F n and define automorphisms A^ and 
Bij by 



-Aij\e k ) 



Bij(e k ) 



^l^J 5 



^2 % ' 



if i = k, 
otherwise, and 

if i = k, 

otherwise. 



Then {Aij, Bij} with i ^ j, generate the index two subgroup of Aut(F n ) 
given by those automorphisms which induce on the abelianization Z" of F n an 
automorphism of determinant one (see, e.g. |Lq]). A straightforward but tedious 
computation shows that relations (i) - (iv) are satisified. o 

We next find some fixed points. 
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Lemma 4.2. Suppose n > 4 and <ft : Aut(F n ) — > DifF + (S' 1 ) is a homomorphism 
with dij — 4>{Aij) and 6y — <p(Bij) where Aij and B^ are the generators of 
Lemma CO. Then each of the difjeomorphisms a^ and bij has a fixed point. 

Proof. We fix i, j and show a^ has a fixed point. Since n > 4 there is an aui with 
{i, j}<~){k, 1} — 0. Let Z(aid) denote the centralizer of aid, so ay £ Z(au)- Also, 
since n > 4 there is 1 < q < n which is distinct from i, j, k, I, so <Xiq, a q j e Z(au)- 



By Lemma \A we know that the rotation number p : Z{au) —* R is a 
homomorphism so /o(ay) = p([ai q , a q j]) = 0. This implies that ay has a fixed 
point, o 

We can now prove the main result of this section 



Theorem h .3L For n > 6, any homomorphisms from KvX{F n ) or or Out(i ? „) 



to Diffi(J) or Diff^S 1 ) /aciors i/irow^/i Z/2Z. 



Proof. Let iJ be the index two subgroup of Aut(F„) from Lemma 4.1. Let 

f + {I) or Diff^ 



be a homomorphism H to Diff^(I) or Diff^S 1 ) and let a^ = 4>{Aij) and 



b^ = <p(Bij) where A^ and B^ are the generators from Lemma 4.1. Then 
{aij, b^} are generators for Image((f>). We will show in fact that the subgroup 
G a generated by {a^} is trivial, as is the subgroup Gb generated by {bij}. The 
arguments are identical so we consider only the set {a^}. Since n > 6, we have 



by Lemma 4.1 that the commutavity graph for the generators {a^} of G a is 



connected. Hence by Corollary 3.6, if these generators are fully supported then 
G a is abelian. But the relations [aij, a~ k } — Ojfc then imply that G a is trivial. 

Hence we may assume at least one of these generators is not fully supported. 
Let J be an interval which is a maximal component of fixed point sets for any 
of the Oj. More precisely, we choose J so that there is a pq with J a component 
of Fix(a p(? ) and so that there is no a^i such that Fix(a / r £ ;) properly contains J. 

We wish to show that J is fixed pointwise by each ay. In case {p, q}ri{i, j} = 
we note that at least the endpoints of J are fixed by a^ be cau se a pq and a.,, 
commute and the endpoints of J are in dFix(a pq ). So Lemma 3^ or Lemma p.3| 
implies these endpoints are fixed by ay . 

For the general case we first show aij(J) fl J ^ 0. In fact there is no ay 
with the property that aij(J) fl J — because if there were then the interval 
J' defined to be the smallest interval containing {a™(J)} ne z is an ay-invariant 
interval with no interior fixed points for a^. (In case G a C Di ff + ( 5 1 ) we need 
the fact that ay has a fixed point, which follows from Lemma [4.2| , in order to 
know this interval exists.) But since n > 6 there is some a^i which commutes 
with both a^ and a pq , and it leaves both J and J' invariant. Applying Kopell's 



Lemma (Theorem 2.1) to aui and a^ we conclude that J' C Fix(aki), which 
contradicts the maximality of J. Hence we have shown that aij(J) fl J ^ for 
alH,j. 



Now by Lemma 4.1 the relation a pq a ap a pq = a q }a pq a a ^ , holds so we may 



-^pg^qp u pq ~ u qp u pq u q p 
apply Lemma 3.7 to conclude that J is fixed pointwise by a qp 



17 



We next consider the generator a pk . If xo is an endpoint of J then since 
ctqk{J) H J 7^ at least one of a g fc(xo) and a~ fc (xo) must be in J. Hence the 
relations [a pq ,a qk ] = a pk x and [dp^a",. 1 ] = a pk imply that a pk (x a ) = x . This 
holds for the other endpoint of J as well, so J is invariant under a pk for all k. 
The same argument shows J is invariant under a g fc for all k. But since a pq is 
the identity on J we conclude from [a P9 , a q k] = a~ k that a p fe is the identity on 
J. Similarly a qk is the identity on J. 

Next the relation a qk a^a qk = o>kq a ik a kq ■> together with Lemma 3.7 implies 



that a kq is the identity on J. A similar argument gives the same result for a kp . 



-l 



Finally, the relation [a,i P , a ■ ] = a.;,- implies that cte is the identity on J. 



pj 



Thus we have shown that any subgroup of Diff,(J) or Diff + (S' 1 ) which is a 
homomorphic image of H , the index two subgroup of Aut(i 7, n ), has an interval 
of global fixed points. In the case of Diff^(S' 1 ) we can split at a global fixed 
point to get a subgroup of Diff + (7) which is a homomorphic image of H. In 
the I case we can restrict the action to a subinterval on which the action has no 
global fixed point. But our result then says that for the restricted action there 
is an interval of global fixed points, which is a contradiction. We conclude that 
the subgroup of G a generated by {a^} is trivial and the same argument applies 
to the subgroup Gb generated by {hj}. So <j>(H) is trivial. Since Aut(F n )/H 
has order two any homomorphism from Aut(F n ) to Diff , (/) or Diff^S* 1 ) has 
an image whose order is at most two. The fact that Diff,(7) has no elements 
of finite order implies that in this case 4> is trivial. 

Since there is a natural homomorphism from Aut(F n ) onto the group Out(F n ), 



the statement of Theorem 1.3 for Out(F n ) holds, o 



4.2 Finite index and other subgroups 



In this section we prove Theorem 1.4. Recall that the Torelli group T g ^ k is the 
subgroup of Mod(g, k) consisting of diffcomorphisms of the fc-punctured, genus 
g surface S 9i fc which act trivially on H (S fl) fc;Z). Of course any finite index 
subgroup of Mod(<7, k) contains a finite index subgroup of T 9tk , so we need only 
prove the thoerem for the latter group; that is, (3) implies (1). 

So, let H be any finite index subgroup of T gyk ,g > 3, k > or of Aut(F„) or 
Out(F„) for n > 6. Suppose that H acts faithfully as a group of C 2 diffeomor- 
phisms of / or S , so that no nontrivial element of H has an interval of fixed 



points (i.e. the action is fully supported, in the terminology of §3.1). In each 
case we claim that H contains infinite order elements a, 6, c with the property 
that, for each r > 2, the elements a r , b r generate an infinite, non-abelian group 
which commutes with c s for all s. 

To prove the claim for T 9ik one takes a, 6, c to be Dehn twists about homo- 
logically trivial, simple closed curves with a and b having positive intersection 
number and c disjoint from both. For r sufficiently large the elements a r ,b r , c r 
clearly lie in H . As powers of Dehn twists commute if and only if the curves 
have zero intersection number, the claim is proved for T g ,k. A direct calcula- 
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tion using appropriate generators Aij as in Lemma 4.1 gives the same claim for 
Aut(F„) and Out(F n ) for n > 2. 



Theorem 1.4 in the case of I now follows immediately from Theorem 3.5 and 
the fact that the group < a r ,b r ,c r > has connected commuting graph but is 
non-abelian. For the case of S 1 , note that if c r has a fixed point for any r > 
so do a r ,b r by Lemma 3.2, so after taking powers we are done by the same 



argument, this time applying Corollary 3.6 



If no positive power of c has a fixed point on S 1 , then c must have irrational 
rotation number, in which case the centralizcr of c in Homeo(S' 1 ) is abelian by 
But < a.b > lie in this centralizcr and do not commute, a contradiction. 



2.10 



The proof of Theorem 1.4 for Aut(F„) and Out(F n ) is similar to the above, 
so we leave it to the reader. 



5 Baumslag-Solitar groups 

In this section we study actions of the groups 

BS(m, n) =< a, b : a& m a _1 = b n > 

For n > m > 1 these groups are not residually finite, hence are not linear (see, 
e.g. pi). 



5.1 An analytic action of BS(m,n) on R 

In this subsection we construct an example of an analytic action of BS(m, n), 1 < 
77i < n on R. It is straightforward to find diffeomorphisms of R which satisfy 
the Baumslag-Solitar relation. 

Proposition 5.1 (Diffeomorphisms satisfying the Baumslag-Solitar relation). 

Let f n : S 1 — > S 1 be any degree n covering and let f m : S 1 — * S 1 be any degree 
77i covering. Let g n and g m be lifts of f n and f m respectively to the universal 
cover R. Then the group of homeomorphisms o/R generated by g — g m gn l an d 
the covering translation h(x) = x + 1 satisfies the relation gh m g~ l = h n . 



Proof. We have 



and 



g- 1 (h n (x))=g- 1 (x + n) 

= g-\x) + l 

= K9n l ( x ))> 
g m (h{x)) = g m (x + 1) 

= 9m(x) +777 

= h m (g m (x)). 
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So 



9(h n {x))=g m {9n\h n {x))) 

= h m (g m (g- 1 (x))) 
= h m (g(x)). 



We will now show how to choose the diffeomorphisms constructed in Proposi- 



tion 5.1 more carefully so that we will be able to use a Schottky type argument 
to show that the diffeomorphisms will satisfy no other relations. Part of the 
difficulty will be that the "Schottky sets" will have to have infinitely many 
components. 

We first construct, for each j > 1, a C°° diffeomorphism Qj of R which is 
the lift of a map of degree j on S . Let a = 1/10. We choose a C°° function 
Qj : (—a/2, 1 + a/2) — > R with the following properties: 

Qj(x) = -x for x <E (-a/2, a/2), 

©jO) = 3 + f (a - 1) for S (a, 1 + a/2), 

0$(a:) >o. 

Note that for x € (—a/2, a/2) we have 6j(x + 1) = ©j(aO + j- It follows that 
we can extend 6^ to all of R by the rule <dj(x + k) = Qj(x) + jk. Thus the 
extended Qj is the lift of a map of degree j on S" 1 . This map is in fact a covering 
map since Qj is a diffeomorphism. 

We shall be particularly interested in the intervals A = [—a, a], A s = 
[-a, 0], A u = [0, a], and C = [a, 1 - a]. Note that [0, 1] = A u U C U (A s + 1). 
Also by construction Qj(A s ) C A s and Qj(C) C (A s + j). More importantly 
we have the following inclusions concerning the union of integer translates of 
these sets. For a set X C R and a fixed io, we will denote by X + i the set 
{x + i \ x e X}, by X + Z the set {a; + i| x £ X, i e Z}, and by X + mZ the 
set {x + mi\ x G X, i e mZ}. 

Suppose fc is not congruent to modulo j. Then we have 

Q 3 {A S + Z) cA s +jZ, 
Q J {C + Z)dA s +jZ, 
Q j (A u +jZ + k) cA s +jZ 
Qj\C + Z) cA u + Z. 

We now define two diffeomorphisms of R. Let g n (x) — Q n (x) and let 
g m (x) = Q m (x - 1/2) + 1/2. We define B = A + 1/2, B s = A s + 1/2, B u = 
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A u + 1/2, and C2 = C + 1/2. Then from the equations above we have 

g n {C + Z) cA s + nZ 
g- 1 (C + Z)cA u + Z 

g- l (A + k)cA u + Zifk<£nZ (1) 

g m {C 2 + Z) CB S +mZ 

g- 1 (C 2 + Z)GB u + Z (2) 

g m l {B + k)cB u + Ziik(£ mZ (3) 

The diffcomorphisms g m and g n can be approximated by analytic diffeomor- 
phisms which still satisfy the equations above and are still lifts of covering maps 
on S . More precisely, the function 4>{x) = g n {x) — nx is a periodic function on 
R and may be C 1 approximated by a periodic analytic function. If we replace 
g n by 4>{x) + nx then the new analytic g n will have postive derivative and be 
a lift of a degree n map on S . If the approximation is sufficiently close, it will 
also still satisfy the equations above. In a similar fashion we may perturb g m 
to be analytic while retaining its properties. 

Note that B + ZcC + Z and A + Z C C 2 + Z, so we have the following key 
properties: 

g n (B + Z)cA s +nZ, (4) 

g- 1 (B + Z)cA u + Z (5) 

g m (A + Z) C B s + mZ (6) 

g-\A + Z)cB u + Z (7) 

Define g(x) = g n (gm ( x )) an d h(x) — x+l, and let G be the group generated 



by g and h. Propostion 5.1 implies that gh m g~ 1 = h n . Hence there is a 
surjective homomorphism $ : BS(m, n) — > G sending b to g and a to h. We 
need only show that $ is injective. 

Lemma 5.2 (normal forms). In the group BS(m,n) with generators b and 
a satisfying the relation ba m b~ 1 = a™ every nontrivial element can be written 
in the form 

a r "6 e "a r "- 1 6 e "- 1 ...a ri b ei a r ° 

where ei — ±1 and n E Z /iave i/ie property that whenever ej = 1 and e,_i = — 1 
we /law r, ^ mZ, and whenever e^ = —1 and ej_i — 1 we have n <£ nZ. 



While Lemma 5.2 follows immediately from the normal form theorem for 



HNN extensions, we include a proof here since it is so simple in this case. 

Proof. If we put no restrictions on the integers r, and, in particular allow them 
to be 0, then it is trivially true that any element can be put in the form above. 
If for any i, a — 1, e%-\ — —1 and r, = ink E mZ then the fact that ba mk b^ 1 = 
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a allows us to substitute and obtain another expression in the form above, 
representing the same element of BS(m, n), but with fewer occurences of the 
terms b and & _1 . Similarly if e^ = — 1, ej_i = 1 and r, = nk £ nZ then 
b~ 1 a nk b can be replaced with a mfc further reducing the occurences of the terms 
6 and b^ 1 . 

These substitutions can be repeated at most a finite number of times after 
which we have the desired form, o 

We now show that BS(m, n) is a subgroup of Diff+(R). Note that Diff+(R) C 
Homeo+(/) C Homeo_(_(iS' 1 ), the first inclusion being induced by the one-point 



compactification of R. Hence the following implies Theorem 1.5 



Proposition 5.3. The homomorphism $ : BS(m, ri) — ► G defined by $(&) = g 
and $(a) — h is an isomorphism. 

Proof. By construction <& is surjective; we now prove injectivity. To this end, 



consider an arbitrary nontrivial element written in the form of Lemma 5.2. After 
conjugating we may assume that r n = (replacing r$ by r$ — r n ). So we must 
show that ($ applied to) the element 



acts nontrivially on R. If n = 0, i.e. if a = h r °, then a clearly acts nontrivially 
if r$ =/= 0, so we may assume n > 1. 

Let x be an element of the interior of C n C2 . We will prove by induction on 
n that a(x) £ (A s + nZ) U (B s + mZ). This clearly implies a(x) 7^ x. 

Let n — 1. Then assuming ei = 1, we have a (a: ) —g(h r °(x)) — g n {g^ {h ra (x))) . 
But x £ C2 + Z implies h r °(x) = x + r £ C2 + Z. So equation (||) implies 
(7~ 1 (x + r ) £ B u + Z, and then equation (0) implies g n (g~ 1 (x + r ) ) G A s +nZ. 
Thus ei = 1 implies a(x) £ A s + nZ. One shows similarly if ei = —1 then 
a(x) =ff(/i r «(x)) G B s +mZ. 

Now as induction hypothesis assume that 



V = g ek h r "- 1 g eh - 1 . . . h ri g ei h ro {a 



and that either 



efc = 1 and y G A s + nZ, or 
e fc = -1 and y £ B s + mZ. 

We wish to establish the induction hypothesis for k + 1. There are four 
cases corresponding to the values of ±1 for each of e^ and ek+i- If e^ = 1 
and e/c+i = 1 then y £ A s + nZ so /i ri (t/) = y + r, G A s + Z and equation 
(0) implies that g^iv + r G i?" + Z. Hence by equation (|), g(h ri (y)) = 
(7„(<7~ 1 (y + r-t)) G A s + nZ as desired. On the other hand if efc = 1 and 
efc + i = —1 then rj; ^ nZ. Hence y £ A s + nZ implies h ri (y) = y + n £ A s + p 
with p £ nZ. Consequently y' — g~ l {y + ri) £ A u + Z by equation (Q). So 
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equation (^|) implies g^ 1 (h Ti (y)) = g m (y') G B s + rriL. Thus we have verified 
the induction hypothesis for k + 1 in the case e^ = 1. 

If ek = -1 and e k+1 = -1 then y e B s + mZ so /i n (y) =i/lrjeB s |Z 
and equation (||) implies that g^ 1 (y + »"i) £ A u + Z. Hence by equation (||) 
g(h Ti (y)) = g m {gn 1 {y + r i)) £ B s + mZ as desired. Finally for the case efc = — 1 
and ek+i = 1 we have r k $. mZ. Hence y G B s + mZ implies h ri (y) = y + n G 
B s + p with p £ mZ. Consequently y' = g^-{y + n) G B u + Z by equation (||). 
So equation (Q) implies g(h ri (y)) = g n (y') G A s + nZ. Thus we have verified the 
induction hypothesis for k + 1 in the case efc = — 1 also, o 

Remark. We showed in the proof of Proposition |5.3| that any element of the 
form 

g ^h rn - 1 g en - 1 ...h ri g ei h ro 

acts nontrivially on R provided e% = ±1, r$ G Z, whenever e$ = 1 and ej_i = — 1 
we have rj ^ mZ, and whenever e^ = —1 and ej_i = 1 we have r$ ^ nZ. As a 
corollary we obtain the following lemma which we will need later. 

Lemma 5.4. In the group BS(m,n) with generators b and a satisfying the 
relation ba m b~ 1 = a n every element of the form 

6 e "a r "- 1 6 e '- 1 ...a ri 6 ei a ro , 

is nontrivial provided e$ = ±1, r, G Z, and whenever e, = 1, e,j_i = —1 we 
have j-j ^ mZ anrf whenever e$ = — 1, ej_i — 1 we have r^ <£ nZ. 

Of course this lemma can also be obtained from the normal form theorem 
for HNN extensions. 

5.2 General properties of BS{m, n) actions 

In this short subsection we note two simple implications of the Baumslag-Solitar 
relation. 

Lemma 5.5. If BS(m,n) acts by homeomorphisms on S 1 with generators g,h 
satisfying gh m g~ l — h n then h has a periodic point whose period is a divisor of 
\n — m\. 

Proof. Consider the group of all lifts to R of elements of BS(m, n) acting on 
S 1 . Choose lifts H and G of h and g respectively and let T(x) = x + 1 denote 
the covering translation. We have 

GH m G -i = H n T p 

for some integer p. Each lift of an element of BS(m, n) has a well defined trans- 
lation number in R and these are topological conjugacy invariants. Hence 

m T {H) = r(H m ) = r(GH m G- 1 ) = r{H n T p ) =p + nr{H). 
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Solving we conclude 

r(H) = -JU 
m — n 

Since H has a rational rotation number h has a point whose period is a divisor 
of \n — m\. o 

Lemma 5.6. Suppose g andh are homeomorphisms o/R and satisfy gh m g~ l = 
h n . Then if h is fixed point free, g has a fixed point. 

Proof. Assume, without loss of generality, that h(x) = x + 1 and n > m and 
g(0) €\p,p+l}. Then 



h nk (g(0)) = (g(0)+nk) e [p + nk,p + nk + 1]. 



But 



h n g = gh m , so 

h nk g = gh mk , and 

h nk (g(0))= g(h mk (0)) =g(mk), so 
g(mk) £ [p + nk,p + nk + 1]. 

For k sufficiently large this implies g(mk) > mk and for k sufficiently negative 
that g(mk) < mk. The intermediate value theorem implies g has a fixed point. 



5.3 C 2 actions of BS(m, n) 

In contrast to the analytic actions of BS(r7i,n) on R, we will show that there 
are no such actions, even C 2 actions, on cither / or S 1 . 

Lemma 5.7. Suppose g and h are orientation-preserving C 2 diffeomorphisms 
of I satisfying gh m g~ 1 = h n . Then h and ghg~ l commute. 

Proof. If x G Fix(/i) then g(x) — g(h m (x)) = h n (g(x)) so g(x) is a periodic 
point for h. But the only periodic points of h are fixed. We conclude that 
ff(Fix(ft)) - Fix(ft). 

Let (a, b) be any component of the complement of Fix(/i) in [0,1]. Since 
g{F\x(hj) = Fix(h) we have that ghg~ x {[a, b\) — [a,b\. 

Let H be the centralizer of h n — gh m g~ l . Note that h and ghg^ 1 are both 
in H. Let / be any element of H. Then if / has a fixed point in (a, b), since 
it commutes with h n we know by Kopell's Lemma (Theorem |2.1| ) that / = id. 
In other words H acts freely (though perhaps not effectively) on (a, b). Hence 
the restrictions of any elements of H to [a, b] commute by Holder's Theorem. 
In particular the restrictions of h and ghg~ l to [a, b] commute. But (a, b) was 
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an arbitrary component of the complement of Fix(/i). Since the restrictions of 
h and ghg~ l to Fix(/i) are both the identity we conclude that h and ghg^ 1 
commute on all of /. o 

As a corollary we have: 

Theorem |1.6| for /: If m and n are greater than 1 then no subgroup o/Diff + (/) 
is isomorphic to BS(m,n). 



Proof. If there is such a subgroup, it has generators a and b satisfying a" = 
ba m b~ 1 . Lemma 5.7 asserts that a and c = bab^ 1 commute. But the commuta- 



tor [c,a] is ba l b 1 abab 1 a and this element is nontrivial by Lemma 5.4. This 
contradicts the existence of such a subgroup, o 

Showing that BS(m,n) has no C 2 action on the circle is a bit harder. 



Theorem 1.6 for S . If m is not a divisor of n then no subgroup ofP)iS + (S 1 



is isomorphic to BS(m,n). 

Proof. Suppose there is such a subgroup, and we have generators g, h satisfying 



gh rn g~ 1 — h n . By Lemma 5.5, h has a periodic point. Let Per(h) be the 
closed set of periodic points of h (all of which have the same period, say p). If 
x G Per(ft.) then g(x) — gh mp — h np g(x) which shows g(Per(h)) — Per(h). 

The omega limit set u(x,g) of a point x under g is equal to a subset of 
the set of periodic points of g if g has periodic points, and is independent of x 
if g has no periodic points. Since g is C 2 we know by Denjoy's theorem that 
uj(g,x) is either all of S 1 or is a subset of Per(g) (see | dMvS| ). In the case 



at hand ui{g,x) cannot be all of S 1 since x G Per(/i) implies ui(x,g) C Per(/i) 
and this would imply h p = id. Hence there exists xq G Per(g) n Per(/i). Then 
xq G Fix(h p ) n Fix(g g ) where q is the period of points in Per(g). 

The fact that xq has period p for h implies that its rotation number under 
h (well defined as an element of R/Z) is k/p + Z for some k which is relatively 
prime to p. Using the fact that p(h m ) = p(gh m g~ 1 ), we conclude that 

m(k/p) + Z = p(h m ) - p(gh m g- 1 ) - p(h n ) - n(k/p) + Z 

Consequently mk = nk mod p and since k and p are relatively prime m = n 
mod p. The fact that 1 < m < n and n — m = rp for some r G Z tells us that 
p (£: mTi because if it were then n would be a multiple of m. Similarly p £ ^Z. 

Let G be the group generated by ho = hP and go — g q . Now G has a global 
fixed point Xq, so we can split S 1 at xq to obtain a C 2 action of G on I. Note 
that gohfgo 1 = 5 «^ m V 9 = h pm " = hg 



We can apply Lemma 5.7 to conclude that /io and gohog commute. Thus 



/i p and c = g q h p g q commute. But their commutator 

[ c , h p ] = g q h~ p g- q h- p g q h p g- q h p . 
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Since p £ rraZ and p £ riL this element is nontrivial by Lemma 5.4. This 
contradicts the existence of such a subgroup, o 



5.4 Local rigidity of the standard action of BS(l,n) 

We consider the action of BS(l,n) on R given by elements of the form x — ► 
n X + b where k £ Z and 6 £ Z[l/n]. It is generated by the two elements 
go(x) = nx and ho{x) = x + 1. 



The following result is essentially the theorem of Shub from [Sh]. We give 
the proof since framing his result in our context is nearly as long. 

Theorem 5.8. There are neighborhoods of go and ho in the uniform C 1 topol- 
ogy such that whenever g and h are chosen from these respective neighborhoods, 
and when the correspondences go — ► g and ho — > h induce an isomorphism of 
BS(l,n) to the group generated by g and h, then the perturbed action is topo- 
logical^ conjugate to the original action. 

Proof. Since g is uniformly C 1 close to go (x) = nx, its inverse is a contraction 
of R and has a unique fixed point, which after a change of coordinates we may 
assume is 0. Since h is uniformly close to ho(x) = x + 1 it is fixed point free and 
after a further change of co-ordinates we may assume h n (0) = n. Consider the 
space H of C° maps <\> : R — * R such that ho<\> — (ph. and h(Q) = 0. Any <f> £ H 
satisfies 4>(n) = n for all n £ Z. Moreover, such a <f> is completely determined by 
its values on the interval [0, 1], since <j)(x + n) = 4>{x) + n. Indeed any C° map 
cj) : [0, 1] — ► R is the restriction of some element of H. Clearly, a sequence in H 
will converge if and only if it converges on [0, 1]. Hence H can be considered a 
closed subset of the complete metric space of continuous functions on [0, 1] with 
the C° sup norm. 

Consider the map F on H given by G(cf>) = go~ 1( t > 9- Then G((f>)h — g^ x <pgh = 
g ~ 1 cj>h n g = go hofig = hog ~ 1 4 i g — hoF{4>). It follows F : H — > H. Moreover 
in the C° sup norm this map is easily seen to be a contraction. It follows that 
there is a unique fixed point 4>o f° r F in H . 

The map <f>Q : R — * R is continuous and satisfies 

/&o0o =4>oh n , and 
9o4>o = M\ 

for all n S Z. The first of these equations implies </>o has image which is 
unbounded above and below, so 0o is surjective. This equation also implies 
that there is a uniform bound on the size of <Pq 1 (x) for x £ R. Indeed if M is 
an integer and M > sup\<j)o(0) — 4>q(x)\ for x £ [0, 1] then y > M + 1 or y < —M 
implies 4>o(y) ¥" < /'o( a; )- So the sets 4>q ( z ) have diameters with an upper bound 
independent of z £ R. In particular, <f>o is proper. 

We can now see that 0o is injective. Since (f>o(y) = <j>o{x) implies go(4>o{y)) = 
go((f>o(x)) which in turn implies 4>o(g n (y)) — <po(g n (x)) and g is is a uniform 
expansion, it follows that if </>o fails to be injective then the sets 4>o ( z ) do not 
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have a diameter with an upper bound independent of z. This contradiction im- 
plies <po must be injective. The fact that 4>q is proper implies 4>q is continuous. 
Hence <j>o is a topological conjugacy from the standard affine action of BS(1, n) 
on R to the action of the group generated by g and h. o 

In contrast to local rigidity of the standard action, Hirsch |B has found real- 
analytic actions of BS(1, n) on the line which are not topologically conjugate to 
the standard action. 

Theorem 5.9 (Hirsch [jH]]). There is an analytic action of BS(l,n) on R 
which is not topologically conjugate to the standard affine action. 

The construction is similar to our construction of an analytic BS(m, n) ac- 
tion on R. The relation ghg~ l — h n is satisfied by analytic diffeomorphisms g 
and h and g has a fixed attracting fixed point. Hence g cannot be topologically 
conjugate to the affine function go(x) = nx. 
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